1. Field of the Invention
The present invention relates to video data processing and, in particular, concerns frame rate conversion of digital video data in circumstances where the video data exhibits frame-wide changes between different frames, such as, for example, frame-wide changes in pixel intensity due to fade in or fade out.
2. Description of the Related Art
Frame rate conversion is a common occurrence with video data processing. Often, the original frame rate at which video data is recorded differs from the frame rate of a subsequent displaying device. In these circumstances, it is often desirable to generate new frames from the original video data in order to match the frame rate of the subsequent displaying device. Often these new frames are inserted or interpolated in between original frames so that the motion of objects within the frames appears smoother and more continuous.
Typically, when a frame is to be generated, or interpolated, the display device will have to determine the luminance of objects or pixels in the interpolated frame. Some objects or pixels are stationary from frame to frame and others are moving. The luminance of any object in the interpolated frame may thus correspond to a stationary object or a motion object. One technique for determining the luminance of objects in the interpolated frame is motion compensated interpolation, a procedure where the relative positions of the moving objects in the original frames are evaluated to determine a motion vector of the moving object which can be used to generate the interpolated frame.
In many circumstances, motion objects are identified in successive frames in the original video data by determining an identifiable characteristic, such as the luminance, of the pixels comprising the moving object in one or more original frames and then looking for a group of pixels having the same or similar characteristic in successive frames. In this way, the vector of the moving objects can be determined in successive frames. Once the vector of the moving objects in successive frames have been determined, this information may be used to determine the luminance of objects in the interpolated frame if they correspond to the vector. In one implementation, a linear regression technique can be used to determine the motion vector of the motion object based upon the two succeeding frames. In other implementations, multiple frames can be analyzed and used to determine the vector of the motion object.
One difficulty that occurs when performing this type of identification of moving objects is that the overall characteristics of the frame, such as luminance, can also vary. This circumstance can particularly occur when the frame is fading in or out, e.g. the overall luminance of all or substantially all of the pixels is changing in the entire frame. When this type of change does occur, pixels or groups of pixels in the successive frames can end up with the same or similar luminance values as unrelated pixels or groups of pixels in the previous frames causing the unrelated pixels or groups of pixels to be identified as the moving object which can lead to an incorrect determination of the motion vector of motion objects and misassignment of the luminance of objects in the interpolated frame. This circumstance is graphically illustrated in FIG. 1.
As shown, motion objects in a previous frame (n−1) have a luminance of 100 and 80 respectively as determined by well-known processing techniques, such as sum of absolute differences (SAD). During fade in or fade out, there may be a uniform loss of luminance of 20 for substantially all of the pixels in the successive frame (n) resulting in the two motion objects having respective values of 80 and 60. In this circumstance, the motion object in the original frame (n−1) having the value of 80 will be assumed to be the motion object which has the value of 80 in the successive frame (n) even though this is actually the motion object that in the preceding frame (n−1) had a value of 100 thus resulting in an erroneous motion vector. Thus, in any new frame that is created between the frames (n−1) and (n), the objects that are intersected by the erroneous motion vector will be assigned the wrong luminance value. This can result in visual distortions and artifacts being introduced into the video stream thereby lowering the quality of the image.
From the foregoing, it will be appreciated that there is a need for improved process for determining the vectors of moving objects in a video stream and determining luminance values of objects in interpolated frames. To this end, there is a need for a better process for determining luminance values in interpolated frames that accounts for global changes in the video images between frames such as the image fading in and fading out.